Second-Order Cohesion

Similarity in contextual behavior between words is considered a source of “lexical cohesion,” which is otherwise hard to measure or quantify. Such contextual similarity is used by an implementation for text segmentation, the VecTile system, which uses precompiled vector representations of words to produce similarity curves over texts. The performance of this system is shown to improve over that of the TextTiling algorithm of Hearst (1997).